Detection of unauthorized user assistance of an electronic device based on the detection or tracking of eyes

ABSTRACT

Methods and systems are provided in this disclosure for operating an electronic device. The electronic device includes a user interface, a camera, and an electronic processor. The electronic processor determines whether a user assistance restriction is established for a current functional operation of the electronic device. The camera captures at least one image, the electronic processor analyzes the at least one image to detect one or more eyes in the field of view of the camera, and the electronic processor detects a violation of the user assistance restriction for the current functional operation based at least in part on the analysis of the at least one image. When a violation of the user assistance restriction is detected, the electronic processor adjusts one or more operations of the electronic device.

BACKGROUND OF THE INVENTION

Some electronic devices are designed to be secured, restricting accessto particular users. For example, a smart phone or tablet computer mayrequire entry of a password in order to access the device or certainfunctionality of the device.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention, and explainvarious principles and advantages of those embodiments.

FIG. 1 is a block diagram of a secured electronic device in accordancewith some embodiments.

FIG. 2 is a front view of a smart phone device configured as the securedelectronic device of FIG. 1 in accordance with some embodiments.

FIG. 3 is a front view of a safe with an electronic locking mechanismconfigured as the secured electronic device of FIG. 1 in accordance withsome embodiments.

FIG. 4 is a flowchart of a method of controlling the operation of thesecured electronic device of FIG. 1 by detecting unauthorized userassistance in accordance with some embodiments.

FIG. 5 is a flowchart of an alternative method of controlling theoperation of the secured electronic device of FIG. 1 by detectingunauthorized user assistance in accordance with some embodiments.

FIG. 6 is a flowchart of a method of detecting unauthorized userassistance by detecting a number of eyes in an image captured by acamera of the secured device of FIG. 1 in accordance with someembodiments.

FIG. 7 is a flowchart of a method of detecting unauthorized userassistance by determining a gaze direction of eyes detected in an imagecaptured by a camera of the secured device of FIG. 1 in accordance withsome embodiments.

FIG. 8 is a flowchart of a method of detecting unauthorized userassistance by tracking movement of a gaze direction of eyes detected ina video stream captured by a camera of the secured device of FIG. 1 inaccordance with some embodiments.

FIG. 9 is a flowchart of a method of detecting unauthorized userassistance by detecting spoken keywords in an audio stream captured by amicrophone of the secured device of FIG. 1 in accordance with someembodiments.

FIG. 10 is a flowchart of a method of detecting unauthorized userassistance by determining a sequence of alphanumeric characters enteredthrough a user interface of the secured device of FIG. 1 and detectingthe same string of alphanumeric characters spoken in an audio streamcaptured by a microphone of the secured device of FIG. 1 in accordancewith some embodiments.

FIG. 11 is a flowchart of a method of detecting unauthorized use of atext entry assistance function in the secured device of FIG. 1 inaccordance with some embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

Methods and systems are provided in this disclosure for operating anelectronic device. The electronic device includes a user interface, acamera, and an electronic processor. The electronic processor determineswhether a user assistance restriction is established for a currentfunctional operation of the electronic device. The camera captures atleast one image, the electronic processor analyzes the at least oneimage to detect one or more eyes in the field of view of the camera, andthe electronic processor detects a violation of the user assistancerestriction for the current functional operation based at least in parton the analysis of the at least one image. When a violation of the userassistance restriction is detected, the electronic processor adjusts oneor more operations of the electronic device.

FIG. 1 is a block diagram of a secure device 100. The secure device 100is an electronic device that includes a secure device electronicprocessor 101 and a secure device memory 103. The secure device memory103 is a non-transitory computer-readable memory that storesinstructions that are accessed and executed by the secure deviceelectronic processor 101 to provide the functionality of the securedevice 100 such as described further below. The secure device 100 alsoincludes a user interface. In the example of FIG. 1, the user interfaceincludes a display screen 105 and an input device 107. In someimplementations, the input device 107 may include a keypad and thedisplay screen 105 may include a graphic display screen or a segmentedliquid crystal display (“LCD”) screen configured to display onlyalphanumeric characters. In other implementations, the user interfaceincludes a touch-sensitive display screen that embodies both the inputdevice 107 and the display screen 105. The input device 107 and thedisplay screen 105 are communicative coupled to the secure deviceelectronic processor 101. The secure device electronic processor 101receives user inputs through the input device 107 and causes the displayscreen 105 to display output information viewable by the user. Thesecure device 100 also includes a camera 109 and a microphone 111.

In the example of FIG. 1, the secure device 100 also includes a wired orwireless transceiver 113 configured to facilitate communication betweenthe secure device electronic processor 101 and one or more remotelylocated systems. In various implementations, the transceiver 113 may beconfigured to provide wireless communications using one or more wirelesscommunications protocols including, for example, cellular phonenetworks, wifi, Bluetooth, or near field communication (“NFC”). In otherimplementations, the transceiver 113 may be configured to providecommunication over a wired interface including, for example, Ethernet.In still other implementations, the secure device 100 may not includeany transceiver 113 and may not be configured to communication with anyexternal device or system.

The secure device 100 of FIG. 1 can be implemented as various differenttypes of secure electronic devices. For example, FIG. 2 shows an exampleof the secure device 100 implemented as a smart phone 200. The smartphone 200 includes a touch-sensitive display 201 as its user interfaceand a front-facing camera 203 configured to capture images of a useroperating the smart phone 200. The smart phone 200 also includes amicrophone 205 positioned to capture audio during telephone calls or tocapture other audio stream data such as described further in some of theexamples below.

FIG. 3 shows another example of the secure device 100 implemented as alocking safe 300. The safe 300 includes a door 301 and an electroniclocking mechanism 303 that includes a 12-button keypad and a display.The electronic locking mechanism 303 is configured to unlock the door301 when a correct sequence of numeric characters is input through itskeypad. The safe 300 is also equipped with a front-facing camera 305positioned to capture images of a user operating the electronic lockingmechanism 303 and a microphone 307 configured to capture audio streamdata.

The secure device 100 is configured to provide user access andfunctionality on a restricted basis. For example, the smart phone 200 ofFIG. 2 may be configured to allow access to certain functionalityprovided by software applications only after an alphanumeric password isentered through the user interface 201. Similarly, the safe 300 of FIG.3 is configured to allow physical access to the interior space of thesafe 300 by opening the door 301 only after a correct sequence ofnumeric characters is entered through the keypad of the electroniclocking mechanism 303. Furthermore, certain software applications orfunctionality operating on the secure device 100 may be designed to beviewed or used by only a single, specific user.

User assistance can occur in a variety of different ways and can, insome circumstances, enable an unauthorized person to gain access to anelectronic device or certain restricted functionality of the electronicdevice. Some examples of types of user assistance that might possible beidentified as unacceptable user assistance for a particular device or afunction of the device may include multiple people viewing the displayscreen 105 at the same time, copying information from the display screen105 to another device or document, copying information from anotherdevice or document into the user interface of the secure device 100, andopening a program or entering information in response to verbalinstructions received from another person. In some situations, thesetypes of user assistance may be considering to be indicative of a riskof an unauthorized user viewing restricted information or attempting togain access to a restricted device. For example, an unauthorized usermay be attempting to coerce an authorized user into telling him thepassword to gain access to the secure device 100.

To help ensure that restricted functionality of the secure device 100 isaccessed and operated only by authorized users, the secure deviceelectronic processor 101 is configured to detect certain types of userassistance and to determine whether such user assistance violates adefined user assistance restriction (i.e., whether the detected type ofuser assistance is authorized or unauthorized). In some implementations,the specific type(s) of user assistance that is authorized or explicitlyunauthorized may be varied depending on the current function of thesecure device 100. As discussed in further detail below, a look-up tablemay be configured and stored on the secure device memory 103 definingthe user assistance restrictions for the secure device 100 or forindividual functional operations of the secure device, for example, byproviding a list of specific types of user assistance that areunacceptable for each of a plurality of functional operations.

FIG. 4 illustrates an example of a method for detecting one or moretypes of user assistance, determining whether a detected type of userassistance violates a user assistance restriction, and operating thedevice based on that determination. While the secure device 100 isoperating, the secure device electronic processor 101 will periodicallycheck for one or more particular types of user assistance at block 401.Some examples of methods for detecting these particular types of userassistance are discussed in detail below. When user assistance isdetected at block 401, the secure device electronic processor 101identifies the current functional operation of the secure device 100 atblock 403 and then accesses a look-up table stored on the secure devicememory 103. In this example, the look-up table includes a list of anyspecific types of user assistance that have been indicated asunacceptable assistance for each of a plurality of different functionaloperations of the secure device. In some implementations, the look-uptable can be configured by the developer of the secure device 100 or aparticular software application running on the secure device 100. Inother implementations, the look-up table can be created and modified bya system administrator to configure the device based on data and deviceaccess rules for a particular organization.

If, based on the identified current functional operation and theaccessed look-up table, the secure device electronic processor 101determines at block 407 that the detected user assistance is anacceptable type of assistance (i.e., does not violate a user assistancerestriction), then the secure device electronic processor 101 continueswith the standard operation of the current function at block 409.However, if the detected user assistance is identified by the look-uptable as an unacceptable type of user assistance, then the secure deviceelectronic processor 101 adjusts the operation of the current functionat block 411.

The precise way in which the secure device electronic processor 101adjusts the operation of the current function may vary depending on theparticular implementation, the specific type of assistance that isdetected, or the specific type of functional operation. For example, insome implementations, if the current functional operation is a passwordentry and the detected user assistance indicates that the enteredpassword was spoken verbally while the password was entered through theuser interface, then the secure device electronic processor 101 mayreject the password and prevent access to the secure device 100. In somesuch implementations, the secure device electronic processor 101 mayalso output a warning that is displayed on the display screen 105 or maytransmit a notification or an “alert signal” to a remote computer systemthrough the transceiver 113 indicating that a suspected attempt atunauthorized access has been detected. In other implementations, thesecure device electronic processor 101 may disable the secure device 100or “black out” the displayed output on the display screen 105 inresponse to detecting a violation of certain types of violations of userassistance restrictions. Similarly, in some implementations, the securedevice electronic processor 101 may be configured to adjust theoperation of the current function by providing limited access to somefunctionality of the device when unacceptable user assistance isdetected. In some implementations, the secure device electronicprocessor 101 is further configured to log a data entry indicating theday, the time, and/or the location of the secure device 100 at the timethat the unacceptable user assistance is detected, so that theinformation might be used later to investigate possible unauthorized useof the secure device 100.

In the example of FIG. 4, the secure device 100 is configured toperiodically monitor for multiple types of user assistance and, inresponse to detecting a condition indicative of such assistance, todetermine whether the detected type of assistance is acceptable.However, in some implementations, the secure device 100 is configured tomonitor for unacceptable assistance only in response to the initiationof a particular functional operation. One example of such a method isillustrated in FIG. 5.

When a functional operation of the secure device 100 is initiatedthrough the user interface at block 501 (e.g., the secure device 100 isturned on, a password entry screen is displayed, a particular softwareapplication is launched, or a request to view a particular type of datais received), the secure device electronic processor 101 accesses thelook-up table stored on the secure device memory 103 at block 503 andmonitors for any particular types of unacceptable assistance identifiedin the look-up table (i.e., violations of a defined user assistancerestriction for the current functional operation) at block 505.

If the secure device electronic processor 101 does not detect any typeof unacceptable user assistance at block 507, then it continues withstandard operation of the current function at block 509. However, if atype of user assistance that has been identified as unacceptable isdetected at block 507, then the secure device electronic processor 101adjusts the operation of the current function at block 511, for example,as discussed above in reference to FIG. 4.

The secure device electronic processor 101 is configured to detectcertain types of user assistance based on various types of informationincluding, for example, data captured by the secure device 100 or inputsreceived through the user interface of the secure device 100. FIGS. 6through 10 illustrate some examples of methods that may be applied bythe secure device electronic processor 101 in some implementations todetect user assistance based on data from the camera 109 or themicrophone 111. In some implementations, the secured device electronicprocessor 101 may be configured to apply one or more of these methods todetect user assistance, for example, at block 401 of FIG. 4 or at block505 of FIG. 5.

The example of FIG. 6 provides a method for detecting when multiplepersons are viewing the display screen 105 of the secure device 100.First, the camera 109 of the secure device 100 captures an image atblock 601. As discussed above in the examples of FIG. 2 and FIG. 3, thecamera 109 is positioned with a field of view that is likely to capturean image of one or more people viewing the display screen 105 (e.g., afront-facing camera on a smart phone). The secure device electronicprocessor 101 then processes the captured image to detect eyes in theimage at block 603. In this example, the secure device electronicprocessor 101 does not apply any face recognition routines to attempt toidentify the user. Instead, the secure device electronic processor 101simply detects eyes in the image and determines whether the imagecaptured by the camera 109 includes more than one pair of eyes at block605 indicating multiple people viewing the display screen 105. If morethan one pair of eyes is detected in the image, the secure deviceelectronic processor 101 determines that user assistance is detected atblock 607. However, if one or fewer pairs of eyes are detected in theimage, then the secure device electronic processor 101 determines thatno assistance is detected by the method of FIG. 6 at block 609.

FIG. 7 illustrates another example of a method for detecting multipleusers viewing the display screen 105 of the secure device 100. In someimplementations, the method of FIG. 7 can be applied instead of or inaddition to the method of FIG. 6. Again the camera 109 captures an imageat block 701 and the secure device electronic processor 101 processesthe image to detect eyes in the image at block 703. However, instead ofonly counting the number of eyes in the image, the secure deviceelectronic processor 101 further analyzes the image to determine whethera gaze direction of each detected eye is directed at the user interfaceof the secure device 100 (i.e., the display screen 105) at block 705. Ifthe secure device electronic processor 101 determines that the number ofeyes that are detected with a gaze direction directed at the userinterface exceeds two at block 707, then it determines that userassistance in the form of multiple users viewing the user interface isdetected at block 709. However, if two or fewer eyes are detected with agaze direction toward the user interface at block 707, then the securedevice electronic processor 101 determines that this type of userassistance is not detected at block 711.

Although, in the examples discussed above in FIGS. 6 and 7, the securedevice electronic processor 101 is configured to analyze still images todetect user assistance, in some implementations, the secure deviceelectronic processor 101 can be configured to analyze a series of imagesor a video stream to detect user assistance. FIG. 8 illustrates anexample of a method in which video stream data is analyzed to detectwhen a user's gaze is repeatedly moving between the display screen 105and another location, which is potentially indicative of a user copyingdata displayed on the display screen 105 or copying data from anothersource into the user interface of the secure device 100. For example,this detectable behavior may be indicative of a user copying a passwordwritten on a piece of paper into a password entry screen.

The camera 109 captures a video stream at block 801 and the securedevice electronic processor 101 processes multiple frames of the videostream to detect eyes in the captured images at block 803. The securedevice electronic processor 101 tracks movements of the eyes insuccessive frames of the video stream to track changes in a gazedirection at block 805. If the secure device electronic processor 101detects that the gaze direction moves repeatedly between the userinterface of the secure device 100 and another location at block 807,then it determines that criteria indicative of possible user assistanceby copying data to or from the user interface of the secure device 100is detected at block 809. However, if such repeated movements of thegaze direction are not detected at block 807, then the secure deviceelectronic processor 101 determines that this type of user assistance isnot detected at block 811.

FIG. 9 illustrates an example of a method for detecting user assistancebased on audio stream data captured by the microphone 111 of the securedevice 100. The microphone 111 captures an audio stream at block 901 andthe secure device electronic processor 101 processes the audio stream todetect spoken words at block 903. The secure device electronic processor101 accesses a stored list of keywords from the secure device memory 103at block 905 and determines whether any of the keywords in the accessedlist are detected in the spoken words of the audio stream at block 907.If so, the secure device electronic processor 101 determines that userassistance in the form of verbal instructions or suggestions provided toan operator of the secure device 100 is detected at block 909. However,if none of the keywords are detected in the spoken words of the audiostream at block 907, then the secure device electronic processor 101determines that this type of user assistance is not detected at block911.

In some implementations, the list of keywords in the example of FIG. 9can be predefined by a manufacturer or developer of the secure device100 or a software application running on the secure device 100. In otherimplementations, the list of keywords can be created and modified by asystem administrator based on the needs and access restrictions of aparticular organization. In still other implementations, the securedevice electronic processor 101 may be configured to continue to updatethe list of keywords based on spoken words that are detected in theaudio stream, for example, while other types of unacceptable userassistance are detected.

In some implementations, such as in the example of FIG. 9 above, thesecure device electronic processor 101 is configured to determine thatassistance is detected when a single keyword from the accessed list isdetected in the audio stream. In other implementations, the securedevice electronic processor 101 is configured to calculate a valueindicative of a weighted probability of user assistance based at leastin part on a number of keywords that are detected in the audio streamand to determine that a violation of the user assistance restriction hasoccurred when the calculated value exceeds a predefined threshold. Instill other implementations, the list of keywords stored on the securedevice memory 103 includes multiple categories of keywords and requiresdetection of one or more keywords from each category before concludingthat user assistance is detected.

For example, in some implementations, the list of keywords may include alist of function keywords and a list of action keywords for each of aplurality of different functional operations of the secure device 100.In some implementations, at least one keyword from the list of functionkeywords and at least one keyword from the list of action keywords mustbe detected in the audio stream data in order for the secure deviceelectronic processor 101 to determine that this type of user assistanceis detected. In some implementations, a function keyword must bedetected before the action keyword in the audio stream in order for thistype of user assistance to be detected.

Accordingly, the general method illustrated in FIG. 9 can be extended tovarious other types of natural language processing to correlate asequence of actions or functional operations on the secure device 100 tostrings of spoken words detected in the captured audio stream. Forexample, a secure device may be configured with a keyword list that isaccessed when a police patrolling software application is launched onthe secure device 100. The keyword list may include function keywordssuch as “public safety,” “patrolling,” and “policing” and the actionkeywords such as “see,” “show,” and “point.” If the captured audiostream includes the spoken words “Wow! You have a patrolling app? Canyou show it to me?” the secure device electronic processor 101 woulddetermine that user assistance is detected because the spoken words ofthe audio stream include at least one function keyword—“Patrolling”—andat least one action keyword—“show.”

In some implementations, the secure device electronic processor 101 isfurther configured to correlate spoken words from the captured audiostream with alphanumeric text entered into the secure device 100 throughthe input device 107. FIG. 10 illustrates an example of a method fordetecting user assistance relating to dictation or verbal instructionsgiven for alphanumeric text entry into the device, for example, duringentry of a password to gain access to the secure device 100.

The microphone 111 captures an audio stream at block 1001 and the securedevice electronic processor 101 processes the audio stream data todetect spoken words at block 1003. The secure device electronicprocessor 101 also monitors alphanumeric text entered through the inputdevice 107 at block 1005. If the alphanumeric text entered through theinput device correlates to spoken words detected in the audio streamdata at block 1007, the secure device electronic processor 101determines that user assistance in the form of verbal dictation orprompting is detected at block 1009. However, if the alphanumeric textentry cannot be correlated to spoken words detected in the audio streamdata, then the secure device electronic processor 101 determines thatthis type of user assistance is not detected at block 1011.

The methods illustrated in FIGS. 6 through 10 are only some examples oftechniques that can be implemented by the secure device 100 to detectspecific types of user assistance. Some implementations may include oneor more of these techniques to detect user assistance. Otherimplementations may utilize other techniques for detecting these orother types of user assistance in addition to or instead of the examplesillustrated in FIGS. 6 through 10.

Furthermore, although the examples illustrated in FIGS. 1, 2, and 3include both a camera 109, 203, 305 and a microphone 111, 205, 307, insome implementations, the secure device 100 may include only a camera109 and not a microphone 111 or, in the alternative, only a microphone111 and not a camera 109. In other implementations, the secured device100 may include both a camera 109 and a microphone 111, but may beconfigured to detect unauthorized user assistance using data captured byonly the camera 109 or only the microphone 111. In still otherembodiments, the secure device 100 may be configured to detectunauthorized user assistance based on other data in addition to orinstead of images or video captured by a camera 109 and audio streamdata captured by the microphone 111.

For example, FIG. 11 illustrates an example of a method that determineswhether certain other types of data entry techniques are used andwhether those data entry techniques have been identified as anunacceptable form of assistance for a particular secure device 100 or aparticular functional operation of the secure device 100. In particular,the secure device electronic processor 101 detects that a “COPY” or“PASTE” function is used on the secure device 100 at block 1101. Thesecure device electronic processor 101 identifies the current functionof the secure device 100 at block 1103 and accesses a look-up tablestored on the secure device memory 103 at block 1105 that identifies anyspecific types of user assistance that have been identified asunacceptable user assistance for the particular function. If the look-uptable indicates that usage of the detected “COPY” or “PASTE” function isnot permitted for the current operation at block 1107, then the securedevice electronic processor 101 determines that unacceptable assistancein the form of an unpermitted COPY or PASTE function is detected atblock 1109. However, if the look-up table indicates that usage of theCOPY or PASTE function is permitted for the current function of thesecure device 100, then the secure device electronic processor 101determines that the detected user assistance is acceptable and continueswith normal operation of the current function at block 1111.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has,”“having,” “includes,” “including,” “contains,” “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a,” “has . . . a,” “includes . . . a,” or “contains . .. a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially,” “essentially,”“approximately,” “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

We claim:
 1. A method of operating an electronic device by detectingunauthorized user assistance, the electronic device including a userinterface, a camera, and an electronic processor, the method comprising:determining, by the electronic processor, whether a user assistancerestriction is established for a current functional operation of theelectronic device; capturing, by the camera, at least one image;analyzing, by the electronic processor, the at least one image to detecta plurality of eyes in a field of view of the camera; detecting, by theelectronic processor, a violation of the user assistance restriction forthe current functional operation of the electronic device based at leastin part on the analysis of the at least one image; and adjusting, by theelectronic processor, one or more operations of the electronic device inresponse to detecting the violation of the user assistance restriction.2. The method of claim 1, wherein detecting the violation of the userassistance restriction includes detecting two or more pairs of eyes fromthe plurality of eyes in the field of view.
 3. The method of claim 1,wherein analyzing the at least one image includes detecting whether agaze direction of each eye from the plurality of eyes in the at leastone image is directed towards the user interface, and wherein detectingthe violation of the user assistance restriction includes detecting thatthe gaze direction of more than two eyes from the plurality of eyesdetected in the at least one image is directed at the user interface ofthe electronic device.
 4. The method of claim 1, wherein capturing theat least one image includes capturing a series of images, whereinanalyzing the at least one image includes detecting changes in a gazedirection of one or more eyes from the plurality of eyes in the field ofview of the camera, and wherein detecting the violation of the userassistance restriction includes determining that the gaze direction ofthe one or more eyes from the plurality of eyes detected in the seriesof images repeatedly moves between the user interface and anotherlocation while a user input is received through the user interface. 5.The method of claim 1, wherein adjusting the one or more operations ofthe electronic device includes transmitting an alert signal to a remotecomputer system in response to detecting the violation of the userassistance restriction.
 6. The method of claim 1, wherein adjusting theone or more operations of the electronic device includes storing a datarecord logging the violation in response to detecting the violation ofthe user assistance restriction.
 7. The method of claim 1, whereinadjusting the one or more operations of the electronic device includesadjusting a displayed output to display a warning message or to blackout the displayed output in response to detecting two or more usersviewing the user interface.
 8. The method of claim 1, wherein adjustingthe one or more operations of the electronic device includes limitingaccess to the one or more operations of the electronic device when theviolation of the user assistance restriction is detected during passwordentry.
 9. The method of claim 1, further comprising calculating a valueindicative of a probability of unauthorized user assistance based atleast in part on the analysis of the at least one image captured by thecamera, and wherein detecting the violation of the user assistancerestriction includes detecting the violation when the value is greaterthan a predefined threshold.
 10. An electronic device comprising: a userinterface; a camera; and an electronic processor configured to determinewhether a user assistance restriction is established for a currentfunctional operation of the electronic device, analyze at least oneimage captured by the camera to detect a plurality of eyes in a field ofview of the camera; detect a violation of the user assistancerestriction for the current functional operation of the electronicdevice based at least in part on the analysis of the at least one image,and adjust one or more operations of the electronic device in responseto detecting the violation of the user assistance restriction.
 11. Theelectronic device of claim 10, wherein the electronic processor isconfigured to detect the violation of the user assistance restriction bydetecting two or more pairs of eyes from the plurality of eyes in the atleast one image captured by the camera.
 12. The electronic device ofclaim 10, wherein the electronic processor is configured to analyze theat least one image by detecting whether a gaze direction of each eyefrom the plurality of eyes in the at least one image is directed towardsthe user interface, and wherein the electronic processor is configuredto detect the violation of the user assistance restriction by detectingthat the gaze direction of more than two eyes from the plurality of eyesdetected in the at least one image is directed at the user interface ofthe electronic device.
 13. The electronic device of claim 10, whereinthe electronic processor is configured to analyze the at least one imageby detecting changes in a gaze direction of one or more eyes from theplurality of eyes in a series of images captured by the camera, andwherein the electronic processor is configured to detect the violationof the user assistance restriction by determining that the gazedirection of the one or more eyes from the plurality of eyes detected inthe series of images repeatedly moves between the user interface andanother location while a user input is received through the userinterface.
 14. The electronic device of claim 10, wherein the electronicprocessor is configured to adjust the one or more operations of theelectronic device by transmitting an alert signal to a remote computersystem in response to detecting the violation of the user assistancerestriction.
 15. The electronic device of claim 10, wherein theelectronic processor is configured to adjust the one or more operationsof the electronic device by storing a data record logging the violationin response to detecting the violation of the user assistancerestriction.
 16. The electronic device of claim 10, wherein theelectronic processor is configured to adjust the one or more operationsof the electronic device by adjusting a displayed output to display awarning message or to black out the displayed output in response todetecting two or more users viewing the user interface.
 17. Theelectronic device of claim 10, wherein the electronic processor isconfigured to adjust the one or more operations of the electronic deviceby limiting access to the one or more operations of the electronicdevice when the violation of the user assistance restriction is detectedduring password entry.
 18. The electronic device of claim 10, whereinthe electronic processor is further configured to calculate a valueindicative of a probability of unauthorized user assistance based atleast in part on the analysis of the at least one image captured by thecamera, and wherein the electronic processor is configured to detect theviolation of the user assistance restriction by detecting the violationwhen the value is greater than a predefined threshold.